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Applicant or Patentee: ARNOLD et al> 

Serial or Patent No> : 

Attorney Docket No. : 6145-33 

Filed or Issued; 

For: MUSICAL INSTRUMENT SYSTEM 

VERIFIED STATEMENT (DECLARATION) CLAIMING SMALL ENTITY STATUS 
(37 C.P.R. 1.9(f) and 1.27 (c) -SMALL BUSINESS CONCERN 

I hereby declare that I am 

[ ] the owner of the small business concern 

identified below 
[X ] an official of the small business 

concern empowered to act on behalf of the concern 

identified below: 

NAME OF CONCER N Van Koeverina Company 

ADDRESS OF CONCERN 11324 Aurora Avenue 

- Des Moines. lA 50322 

I hereby declare that the above identified small business concern 
qualifies as a small business concern as defined in 13 C^F.R, 
121.12, and reproduced in 37 C.F.R. l,9(d), for purposes of paying 
reduced fees under section 41(a) and (b) of Title 35, United States 
Code, in that the number of employees of the concern, including 
those of its affiliates, does not exceed 500 persons. For purposes 
of this statement, (1) the number of employees in the business 
concern is the average over the previous fiscal year of the concern 
of the persons employed on a full-time, part-time or temporary 
basis during each of the pay periods of the fiscal year, and (2) 
concerns are affiliates of each other when either, directly or 
indirectly, one concern controls or has the power to control the 
other, or a third party or parties controls or has the power to 
control both. 

I hereby declare that rights under contract or law have been 
conveyed to and remain with the small business concern identified 
above with regard to the invention, entitled 

MUSICAL INSTRUMENT SYSTEM 

by inventors Rob C. Arnold, David Van Koevering, Robert D. Lawson, 
Lance E. Lyda, Kenneth R. Noyce, Robert Phillips, John M. Pursey, 
Gary L. Snethen and Warren W. Westlund, II 

described in 

[ X] the specification filed herewith 
[ ] application serial no. 

filed . 

[ ] patent no. ^ ^ 

issued . 

If the rights held by the above identified small business colacern 
are not exclusive, each individual, concern or organization having 
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rights^ to the invention is listed below* and no rights to the 
invention are held by any person, other than the inventor, who 
could not qualify as a small business concern under 37 C.F.R. 
1.9(c) or by any concern which would not qualify as a small 
business concern under 37 C.F.R. 1.9(d) or a nonprofit organization 
under 37 C.F.R. 1.9(e). *NOTE: Separate verified statements are 
required from each named person, concern or organization having 
rights to the invention averring to their status as small entities. 
(37 C.F.R. 1.27) 

NAM E 

ADDRES S 

[ ] INDIVIDUAL [ ] SMALL BUSINESS CONCERN [ ] NONPROFIT 

ORGANIZATION 

NAM E 

ADDRESS — 
[ ] INDIVIDUAL [ ] SMALL BUSINESS CONCERN [ ] NONPROFIT 

ORGANIZATION 

I acknowledge the duty to file, in this application or patent, 
_ notification of any change in status resulting in loss of 
^ entitlement to small entity status prior to paying, or at the time 

of paying, the earliest of the issue fee or any maintenance fee due 
y^. after the date on which status as a small entity is no longer 

appropriate. (37 C.F.R. 1.28(b)) 

fi I hereby declare that all statements made herein of my own 

ru knowledge are true and that all statements made on information and 

D belief are believed to be true; and further that these statements 

=: were made with the knowledge that willful false statements and the 

O like so made are punishable by fine or imprisonment, or both, under 

III section 1001 of Title 18 of the United States Code, and that such 

ry willful false statements may jeopardize the validity of the 

m application, any patent issuing thereon, or any patent to which 

,|| this verified statement is directed. 

NAME OF PERSON SIGNING Kenneth R. Novce 

TITLE OF PERSON OTHER THAN OWNE R President 

ADDRESS OF PERSON SIGNIN G 3401 Scenic Vallev Drive 

/' / ) West Des Moines, lA 150265 

SIGNATURE -A ■ DATE nll^n 'j 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
In re Application of ARNOLD et al . 
Serial No. Examiner: 
Filed: (Herewith) Group Art Unit: 

For: MUSICAL INSTRUMENT SYSTEM 

PRELIMINARY AMENDMENT 

Box Patent Application 

Assistant Commissioner for Patents 

Washington, D.C. 2 0231 

|jj Sir: 

H: Prior to examination on the merits, please amend the above- 

[y identified application as follows: 
L IN THE TITLE : 

m Throughout the application, original Declaration excepted, 

i please amend the title to read --ELECTRONIC MUSIC INSTRUMENT 

SYSTEM WITH MUSICAL KEYBOARD--. 

TN THE SPECIFICATION : 

Cross-Ref erence to Related Application : 

Please delete the cross-reference language on page 1, lines 
2-5 and replace with --This application is a Continuation of 
United States Application No. 08/879,678, filed June 23, 1997, 

WP/ 1417 47 
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now U.S. Patent No. 5,908,997, which claims the benefit of United 
States Provisional Application No. 60/020,601, filed June 24, 
1996, and United States Provisional Application No. 60/021,522, 
filed July 11, 1996, which benefits are also claimed by this 
application. - - 

Page 27, line 4, before ''decoders" insert --bit--. 

REMARKS 

The foregoing Preliminary Amendment is made to correct the 
title of the invention and to add the cross-reference language to 
the related application. No new matter is added. Examination on 
the merits is respectfully requested. 



Respectfully submitted. 



Date : 
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Joseph W. Bain 
Registration No. 34,290 
Scott D. Paul 
Registration No. 42 
Quarles & Brady LLP 
Esperante Building, 
222 Lake view Avenue 
Post Office Box 3188 
West Palm Beach, FL 33402-3188 
Telephone: (561) 653-5000 
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MUSICAL INSTRUMENT SYSTEM 
CROSS-REFERENCE TO RELATED APPLICATIONS 
This application claims the benefit of United States 
Provisional Application No. 60/020,601, filed June 24, 1996, and 
United States Provisional Application No. 60/021,522, filed July 
5 11, 1996. 

FIELD OF THE INVENTION 

The invention relates to electronic musical instruments. 
The invention relates more particularly to electronic musical 
instruments having a computer-based control system. 
10 BACKGROUND OF THE INVENTION 

J Known musical keyboard systems utilizing computer systems 

Jr: -for computer-assisted production of musical output typically have 
^ complicated control systems and user controls, such as numerous 
lU buttons, knobs, switches and sliders, making them impractical for 
15=^^ novice users and even for proficient musicians during 
lH performances. 

IB Early developments in computer-assisted musical instruments 

^ have utilized MIDI (Musical Instrument Digital Interface) 

protocol to process musical instrument inputs for the emulation 
2 0 of musical sound through playback of digitally stored voice 

samples. Numerous devices have been developed utilizing MIDI and 
associated technology to provide electronic keyboards as well as 
digital and digital/acoustic pianos. 

Resulting systems have interfaced with the user in one of 
25 two extremes. The systems either provide little or no 

interaction between the user and the underlying computer system, 
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as in the case of consumer digital pianos, cominonly used for 
player piano and recording functions, or are primarily intended 
for use by recording and performing professionals, utilizing 
complicated interfaces involving numerous buttons, knobs, 
switches and sliders. Other systems utilize complicated visual 
interfaces, making the features largely inaccessible to novice or 
consumer users and unmanageable for "real time" adjustment during 
performances by professional musicians. 

Moreover, most systems incorporating computer-based 
management of a musical instrument have primarily focused on the 
operation of the musical instrument and have- not incorporated the 
musical instrument and its sound production into a larger music 
information management environment that is readily accessible to 
both novices and performing musicians through a consolidated, 
rapidly accessible graphical user interface. Existing systems 
also do not provide a vehicle to launch multimedia applications 
from a consolidated graphical user interface for use in a musical 
instrument and multimedia music information environment. 



SUMMARY OF THE INVENTION 
It is accordingly an object of the invention to provide an 
electronic musical instrument system having a consolidated, user- 
friendly control for substantially all functions and parameters 
of music information management and operation. 

It is another object of the invention to provide an 
electronic musical instrument system that facilitates rapid, 
real-time control adjustment of musical parameters and functions 
to quickly alter the sound of the system by the user, even during 
performance on the musical instrument. 

It is a further object of the invention- to provide an 
electronic musical instrument system that is capable of 
interactive control and response with the musical instrument and 
music information sources for teaching and training. 

It is a still further object of the invention to provide an 
electronic musical instrument system that can obtain digital and 
analog information from a variety of sources and present the 
information to the user in full motion pictures using MPEG-1 
standard video and multi-color graphics or use the information to 
effect automatic playing of the musical instrument by the system 
itself. 

It is yet another object of the invention to provide a 
musical instrument and multiple music information source 
environment for launching and running music related applications 
through a readily accessible graphical user interface. 



These and other objects of the invention are achieved by a 
computer-managed electronic musical instrument system including a 
musical instrument, such as musical keyboard, interactively 
managed by a computer control system that is operated by the user 
5 primarily through a graphical user interface, preferably embodied 
in a touch controlled video display. 

The musical instrument system can further include components 
to obtain digital information from a variety of sources, 
including hard drive, floppy disk, optical disk, and on-line 
10 services and databases through a modem. Thus, the system can 
function to automatically play the musical keyboard, present 
y audio-visual information such as sheet music with performance 
W examples, karaoke, educational and other audiovisual information 
H= and presentations including motion pictures, and produce audio 
ISfiJ and visual output of digital music, text or other information 
downloaded from on-line sources through the modem, 
m According to an aspect of the invention, the system is 

l"^ 3 

^ 

arranged so that all these functions are substantially controlled 
through a graphical user interface, preferably manipulated by 

2 0 input through a touch screen overlay. Thus, a complete music 
information environment is created in conjunction with a 
versatile musical keyboard to enhance the performance and the 
utility of the musical keyboard, controlled through a unified, 
simple, user-friendly interface that can be readily mastered by a 

25 novice and yet facilitates rapid mid-performance adjustments and 
control by a seasoned musician. 
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An electronic music instrument system, in accordance with an 
inventive arrangement, comprises: an electronic music 
instrument, having selectable groups of reproducible sounds and 
individually selectable reproducible sounds, within the groups of 
sounds; an audio signal generator for energizing an audio 
amplifier responsive to different ones of a plurality of digital 
audio sources, including the groups of sounds and the individual 
sounds of the electronic music instrument; a graphical user 
interface for displaying at least one control graphic 
representing controllable parameters of the audio signals 
generated by the generator; and, control means responsive to 
operation of the control graphic for adjusting the controllable 
parameters of the generator and for selectively coupling 
different ones of the sources to the generator* 

The graphical user interface can comprise: a video display; 
and, a touch-responsive overlay. 

In a presently preferred embodiment, the controllable 
parameters include multiple instrument sound selection and sound 
layer assignment, controlled responsive to operation of the at 
least one control graphic; and, the at least one control graphic 
is displayed on a single screen of the graphical user interface. 



BRIEF DESCRIPTION OF THE DRAWINGS 

A more thorough understanding of the invention can be gained 
from a reading of the following detailed description of various 
embodiments of the invention in* conjunction with the accompanying 
drawings, in which: 

FIG. 1 is a perspective view of an embodiment of an 
interactive musical instrument system according to the invention; 

FIG, 2 is an organizational block diagram of an exemplary 
music information management system (MIMS) according to the 
invention; 

FIG. 3 is a block diagram of an exemplary information 
processing subsystem architecture of the MIMS depicted in FIG. 2; 

FIG. 4 is a block diagram of an exemplary conversion board 
for interfacing the user interface display and touch control with 
the MIMS; 

FIG. 5 is a block diagram of an exemplary MIDI sound 
generation subsystem architecture of the MIMS depicted in FIG. 2; 

FIG. 6 is an exemplary software hierarchy for the musical 
instrument system of the invention; 

FIG. 7 is an exemplary graphical user interface control 
screen of the invention for single instrument sound or voice 
selection; 

FIGS. 7A is a preferred graphical user interface control 
screen for metronome functions; 



FIG. 8 is an exemplary graphical user interface control 
screen of the invention for instrument ensemble sound or voice 
selection; 

FIG. 9 is an exemplary graphical user interface control 
screen of the invention for single screen keyboard range 
allocation for the instrument ensemble; 

FIG. 10 is an exemplary graphical user interface control 
screen of the invention for single screen volume adjustment for 
the instrument ensemble; 

FIG. 11 is an exemplary graphical user interface control 
screen of the invention for single screen key transposition 
adjustment for the instrument ensemble; 

FIG. 12 is an exemplary graphical user interface control 
screen of the invention for single screen pan adjustment for the 
instrument ensemble; 

FIG. 13 is an exemplary graphical user interface control 
screen of the invention for drum sound selection; 

FIG. 14 is an exemplary graphical user interface control 
screen of the invention for syncopated style selection and 
control; 

FIG. 15 is an exemplary graphical user interface control 
screen of the invention for music and audio player control; 

FIG. 16 is an exemplary graphical user interface control 
screen of the invention for music and audio player control as 
shown in FIG. 15 with the appearance of a lyrics screen; 



FIG. 17 is an exemplary graphical user interface control 
screen of the invention for multiple layer sequence recording; 

FIG. 18 is an exemplary graphical user interface control 
screen of the invention for setting clock and key sensitivity 
levels; 

FIG. 19 is an exemplary graphical user interface control 
screen of the invention for foot pedal settings; 

FIG. 20 is an exemplary graphical user interface control 
screen of the invention for input settings; 

FIG. 21 is an exemplary graphical user interface control 
screen of the invention for printer selection; 

FIG. 22 is an exemplary graphical user interface control 
screen of the invention for screen saver settings; 

FIG. 23 is an exemplary graphical user interface control 
screen of the invention for file management functions; 

FIG. 24 is an exemplary graphical user interface control 
screen of the invention for copying files from one location to 
another ; 

FIG. 25 is an exemplary graphical user interface control 
screen of the invention for running applications within the 
musical instrument system environment; 

FIG. 26 is an exemplary graphical user interface control 
screen of the invention for saving settings established through 
various control screens; and 
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FIG. 27 is an exemplary graphical user interface control 
screen of the invention for retrieving settings previously 
established. 



DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION 
The invention is directed to an electronic musical 
instrument utilizing a computer-based music information 
management system to process and produce sound and other music 
5 information from a plurality of sources. Generally, the 

invention provides the convergence and integration of various 
subsystems to arrive at an interactive musical instrument system. 
The system can thus derive music and other information from a 
variety of information sources to enhance the performance and 
10 uses of an electronic musical keyboard. The system preferably 
creates a user-friendly environment, established by a 
;J consolidated, touch screen control interface.* 

The preferred embodiments of the invention provide a 
multimedia platform, including components for presenting full 
151^ motion digital video, animation, multi-colors, high resolution 
1. graphics, and digital audio. The software drivers for the 

musical instrument system preferably present motion picture 
ffl through the MPEG-1 standard of 30 frames per second to provide 
B full motion digital video. 
2 0 The musical instrument system preferably includes a storage 

subsystem comprising 720 megabytes through an optical disJc, 500 
megabytes or more in a hard drive, and 1.44 megabytes on floppy 
disk. The optical disk and associated drivers enable use of 
compact discs in a variety of formats, including CD-DA and CD- 
25 ROM. 
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The musical instrument system preferably utilizes MIDI 
(Musical Instrument Digital Interface) standard protocol for 
communications between the musical keyboard and a MIDI sound 
generator, as well as for receipt and transmittal of music 
information between the various digital sources of the system. 
The musical instrument system can further include a modem 
for exchanging information with a variety of on-line sources. 
Through the on-line sources and the optical disk drive, the 
musical instrument system can obtain information for performing a 
variety of functions previously available only through 
complicated or separate systems. 

Referring to the figures and particularly to FIG. 1, an 
embodiment of the musical instrument system of the invention can 
generally include a musical keyboard 10 encased in a grand piano 
housing 12. Alternative housings, including upright, console and 
the like, can be used. The musical keyboard 10 is preferably an 
8 8 -key, velocity-sensitive keyboard that is capable of producing 
electric signals suitable for scanning and conversion into MIDI 
data by a computer system. The system preferably includes a 
plurality of digital information sources, such as a drive 14 for 
a floppy disk 15 and a drive 16 for a compact disk 17, each being 
enclosed in the piano housing 12 and optionally hidden behind a 
cover, such as a hinged door 18. 

According to an aspect of the invention, the control of the 
musical instrument system sound parameters and information 
retrieval and utilization are preferably unified in a graphical 
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user interface presented on a video monitor, such as a flat 
screen display 20 that is preferably equipped with a touch 
control overlay 22 for detecting and processing player input 
through relative position of contact with the overlay 22. The 
flat screen display 20 is preferably positioned flush in a music 
stand 24 of the piano housing 12, although other mountings are 
possible. 

A music information management system ("MIMS" hereinafter) 
for coordinating the interaction between the consolidated 
graphical use interface, the musical keyboard and the plurality 
of digital information sources is preferably encased within the 
housing 12. An overview of an embodiment of the MIMS is set 
forth in FIG. 2. An exemplary design of the music information 
processing subsystem is illustrated in FIG. 3 and FIG. 5 with a 
detail of an exemplary MIDI sound generation subsystem shown in 
FIG. 5. An overview of a touch-sensitive LCD conversion board 
assembly is shown in FIG, 4. Throughout the figures^ effort has 
been made to use the same reference numbers for the same or 
similar components to provide continuity among the illustrated 
embodiments . 

Referring to FIG. 2, the MIMS 26 can generally include a CPU 
subsystem 28, a display subsystem 30, a user input subsystem 32, 
a multimedia audio subsystem 34, an input/output subsystem 36 and 
a MIDI audio subsystem 38. The various subsystems can be linked 
through a bus subsystem 40. The CPU subsystem 28 can include a 
primary processor 42 for the MIMS 26, as well as ROM 44 for 
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storing BIOS instructions and RAM 46 for application execution. 
The CPU subsystem 28 can be supported by DMA/ interrupt 
controllers 48 and components 50 for timers and real time clock 
functions. 

The display subsystem 3 0 preferably includes a graphics 
decoder 52 for generating video images under command from the CPU 
subsystem 28. The functions of the graphics decoder 52 can be 
enhanced by a graphics accelerator 54 for providing improved 
MPEG-1 level video. The graphics decoder 52 can further be 
supported by dedicated video memory 56. The data generated by 
the graphics decoder 52 can be further processed by a digital to 
analog converter 58 for presentation to a display subsystem 59 
including the flat display 20 (FIG. 1) . 

The user input subsystem 32 allows the user to navigate 
through the various control screens of the graphical user 
interface discussed more fully below. The user input subsystem 
32 can provide input controllers 60 for processing user inputs 
through the preferred touch screen overlay 22 (FIG. 1) . 
Alternative user inputs, such as a mouse 62, can also be 
interfaced with the user input subsystem 32. 

According to the invention, a plurality of sources of 
musical digital information are provided through the input/output 
subsystem 36. These information sources can include the CD 
player 16, capable of reading data from compact discs and 
producing CD-DA audio output. The information sources can 
further include a hard drive 64 and the floppy disk drive 14. 
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The information sources can be interfaced with the overall system 
through various controllers 6 6 with CD audio output 68 being 
directed to the multimedia audio subsystem 34, The input/output 
subsystem 3 6 can provide communication components including a 
modem 70 for the transmission and receipt, through telephone 
lines 71, of musical digital information and the generation of 
audio for direct processing by the multimedia audio subsystem 34. 
The communication components can further include a parallel port 
72, an RS-232 serial port 74, and MIDI IN, OUT and THRU ports 76 
interfaced with the system through an I/O controller 78. 

The multimedia audio subsystem 34 is preferably constructed 
to process audio signals from a variety of digital and analog 
sources throughout the MIMS 26. The multimedia audio subsystem 
34 can include a multiple channel stereo mixer 80 for receiving 
and mixing signals from the CD-DA source 68, the modem 70, PCM 
files on line 82, an external microphone 84 and a MIDI sound 
generator 86 within the MIDI audio subsystem 38. 

The analog signals of the microphone 84 can be routed 
directly to the stereo mixer 8 0 or can be further converted to 
digital code through a PCM encoder 88 for further use, 
recordation or other processing by the MIMS 26. PCM files on 
line 82 can be converted to analog form by a PCM decoder 90 
within the multimedia audio subsystem 34. The output of the 
audio subsystem 34 is directed to a sound output assembly 92 
including, for example, an audio amplifier and one or more 
associated loudspeakers . 
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The MIDI audio subsystem 38 provides a MIDI sound module 86 
for generating analog audio signals based on MIDI instructions 
received from various components of the MIMS 26 • Instructions 
for the MIDI sound generator 86 'are managed by a MIDI co- 
processor 94, which also serves to control scanning of MIDI input 
devices and particularly the musical keyboard 10 (FIG. 1) by a 
MIDI scanner 96. The MIDI scanner componentry 96 can be adapted 
to monitor and process input from other MIDI sources, including 
pedals 98, pitch bend and modulation controls 100 and aftertouch 
features 102 in the keyboard 10 • 

Control Processing Unit Subsystem 

Referring to FIG, 3, the music information processing 
subsystem 104 of the invention preferably includes an Advanced 
Micro Devices 586-133 central processing unit 42, processing at 
133 MHz- Alternative central processing units can be utilized 
but the specifications of the preferred unit are considered 
minimum for suitably quick performance. The music information 
processing subsystem 104 preferably includes components to 
establish multimedia capabilities. The music information 
processing subsystem 104 further preferably includes 8 MB of 
application RAM 46 and a 72 pin SIMM socket (not shown) for 
additional memory expansion capability. 

A preferred system BIOS ROM 44 includes standard BIOS 
routines, such as established in Chips and Technologies BIOS 
SE4041, including routines for testing system hardware, a 
bootstrap loader to load the operating system off the hard drive 
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64, and a group of device drivers that service the system I/O 
components, such as the display 20 (FIG. 1), the hard drive 64 
and floppy disk drive 14, a real-time clock, and parallel and 
serial ports. The BIOS preferably additionally includes routines 
5 for booting the system without displaying any diagnostic text on 
the display 20 (FIG. 1) or displaying any operating system boot 
screen. The BIOS can also preferably include conventional system 
debug and test routines to allow the MIDI co-processor 94 (see 
FIG. 5 also) to diagnose problems in the main processor portion 
10 of the system 104. 

The BIOS preferably has a fixed configuration and will not 
J rely on the CMOS RAM, but rather is preferably stored in Flash 
^ memory that can be updated without replacing the device. The 
BIOS can preferably be augmented to include routines to avoid 
15;^:: error messages and other interrupting displays to the user. Some 
hardware test routines can be modified to ignore certain errors, 
such as keyboard error and CMOS RAM Checksum tests. Clock 
tS functions are preferably set via the operating system. 
;;Q The music information processing subsystem 104 further 

2 0 includes components for timing, DMA 48 and interrupt controllers 
50. These functions can be accomplished, for example, by a 
CS4 045 from Crystal Semiconductor, but could be met by any 
chip(s) capable of handling the system's timers and real-time 
clock and optionally implementing the system's DMA and interrupt 
25 controllers. 
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Graphics Subsystem 
The main processor CPU 4 2 is operatively linked through a 
local CPU bus 108 to a plurality of support devices, including 
the graphics subsystem 3 0 (FIG. -2). The graphics subsystem 30 
5 provides a graphics controller 52 that preferably includes a 

built-in graphics accelerator and a digital to analog converter 
(DAC) capable of generating SVGA graphic output, A graphics 
processor providing preferred specifications is the 65548/65550 
Flat Panel CRT GUI DAC Multimedia Engine from Chips and 
10 Technologies, but other similarly configured graphics processors 
can be used. The video output is preferably- directed to the flat 
yp screen display 20 (FIG. 1) , such as a Sharp Model No. LQ10D031 4- 
f{j bit color LCD display. The graphics processor 52 preferably 
provides 640 x 480 resolution at 16-bits/pixel color and 
151^ interfaces with dedicated display memory 56, which is preferably 
JL. 1 megabyte (MB) of EDO high perf ointiance RAM. 
Jf^ The interface to the display 20 and the touch-sensitive 

S overlay 22 (FIG. l) is implemented through a display link 110, 
^fi which can be constructed in conventional manner. The display 
2 0 link chip set preferably takes 20 bits of TTL parallel data from 
the graphics controller 52 and converts it to serial Low Voltage 
Differential Signaling ("LVDS") data that is 3 bits wide. This 
■ conversion occurs at seven times the dot clock rate by way of an 
internal phase lock loop (PLL) . 20 bits of data include 6 bits 
25 red, 6 bits green, 6 bits blue, an H sync, and a V sync. One 

LVDS signal is also used for clocking and sync of the serial data 
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for a total of four LVDS pairs. Referring to FIG. 4, the 
receiving display link 117 converts the serial LVDS data back to 
twenty bits parallel, plus the dot clock to connect directly into 
the LCD flat panel 20. 

Again in FIG. 3, the graphics subsystem further preferably 
includes a graphics accelerator 54 (FIG. 2) capable of YUV-to-RGB 
conversion and image scaling for higher performance MPEG-1 video. 
The MPEG-1 images can be generated through known hardware and/or 
software systems. These functions can be provided, for example, 
by the 65550 chip for the graphics controller 52, the graphics 
accelerator 54, and the DAC 58 (FIG. 2). 

I/O Subsystem 

The music information processing subsystem 104 preferably 
includes a controller 66, such as provided by a CS4045 chip from 
Crystal Semiconductor, which is a multi-purpose peripheral and 
memory interface that can communicate via IDE bus masters 112 
with the hard drive 64 and the CD-ROM drive 16. The CD-ROM drive 
16 is preferably at least a quad-speed (4X) CD-ROM drive, capable 
of 600 KB/sec reads, and the hard drive 64 preferably has at 
least a 540 MB capacity. 

The controller 66 can also include a CS4041 chip used to 
handle addressing and refresh of the main system memory 46 and 
addressing of the system BIOS stored in the ROM 44. 

The controller 66 also preferably provides communications 
with the touch screen overlay 22 (FIGS. 1 and 4) as a standard 
input device through a bi-directional link 114 via a display 
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connector 116, primarily to receive input signals from the touch 
control overlay 22 but also to direct calibrating signals and 
other information to the overlay 22 and its controller. 

Display • Subsystem 
As shown in FIG« 4, the musical instrument system provides a 
user input means which preferably includes the touch panel or 
overlay 22 and an LCD conversion board 118 mounted behind the 
flat panel display 20 (shown schematically) . The arrangement of 
the conversion board 118 for the display and touch screen 
interface can be designed in conventional manner to generate 
screen images pursuant to instructions from the music information 
processing subsystem 104 and to retrieve input signals generated 
by player contact with the touch screen overlay 22. 

The conversion board 118 can be linked to a power supply 120 
arranged to provide dimming control, DC to AC conversion and a 
+12 V supply. A standard voltage regulator 122 for conversion to 
a +5V level can also be provided. The conversion board 118 can 
be linked to the flat panel display 2 0 through a conventional 
thirty-one conductor flat cable 124. As discussed earlier, the 
flat panel display link 117 converts signals received through a 
display connector 116 for further processing and display on the 
flat panel display 20. 

The User Input Subsystem 
A touch screen controller 12 6 preferably includes a Motorola 
68HC05 Microprocessor, a TI TLC1543 10 bit analog to digital 
converter, allowing 1024 points on both the X and Y axis, and a 
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National 93C4 6 EEPROM. The touch screen controller 12 6 controls 
the analog to digital conversion and the brightness of the 
backlight • It also provides serial communication with the main 
processor 42 (FIG. 3), stores and retrieves data in the EEPROM, 
and runs diagnostics. The touch screen controller 126 is 
operatively connected to the touch panel 22, which can be either 
4 or 8 wire analog resistive. 

Because of tolerance variations in both the touch screen 22 
and the interface components, it is necessary to calibrate the 
controller 126. Calibration data is preferably stored in an 
EEPROM so that it is retained after power-down. 

Referring again to FIG. 3, the input/output controller 78 
preferably includes a 37C665 Super I/O chip. The chip is an 
integrated controller that provides interface to the preferred 
3.5" 1.44 MB floppy diskette drive 14, a bi-directional parallel 
port 92, and a serial port 128 for connecting, for example, an 
optional mouse input device (not shown) . 

The subsystem 104 can further provide the modem 70, 
preferably a Rockwell No. SMV144AC modem, capable of transferring 
data at 14400 baud, although slower modem speeds to 9600 baud can 
be used. The modem 70 can be used to access music information 
from on-line services and other remote sources for storage and 
processing in the MIMS. The modem 70 can also enable 
transmission from the system for transferring music information 
to other systems. The modem 70 is connected through a Data 
Access Arrangement (DAA) module 131, which provides electrical 
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isolation and audio connection between the telephone line 71 
(FIG. 2) and the data/modem circuits. The DAA module is designed 
to meet or exceed the FCC part 68 requirements allowing direct 
connection into the Public Switched Telephone Network (PSTN) . 
5 The DAA module 131 provides ring detection and hook switch 

control as well as an audio path and 1500 volt isolation/800 volt 
surge protection. The modem 70 and DAA module 131 interface with 
the telephone line 71 through a standard RJll port 133. 

Multimedia Audio Subsystem 
10 Referring to FIGS. 2 and 3, the multimedia audio subsystem 

preferably includes components for receiving^ and mixing audio 
^13 signals from a plurality of sources including the modem 70, the 
111 MIDI sound generator 86, the microphone 84 with the amplifier 
,g 130. Optionally, alarms or bells from controllers 48, 50 can 
15g also be made audible. The audio output signals are directed to 
an amplifier 134 and loudspeakers 13 6 as well as a headphone 
amplifier and port 138. The system preferably includes a 
headphone present switch 140. The system can optionally provide 
^ volume and balance controls (not shown) . 
20 The multimedia audio system preferably includes a CS4232 

integrated chip which includes the stereo mixer 80, capable of 
combining the multiple signals into a two channels stereo output; 
attenuators for the input signals, a voice encoder 88 (FIG. 2) to 
process the microphone vocal input; and a balance control. 
25 Preferably, there are five possible audio sources in the system: 
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CD Audio, MIDI synthesis, microphone audio, PCM audio and modem 
audio. 

When playing a CD-DA disc in the CD drive 16, the built-in 
Red book, 16-bit audio decoder 13 2 on the CD-ROM drive 16, 
capable of playing CD-DA audio at 44.1 kHz, decodes the CD audio 
to an analog audio output which connects to the CS4232 audio 
mixer and processor 80 through an analog input for mixing, 
processing and output to the audio amplifier 134 and loudspeakers 
13 6 or the headphone amplifier and port 13 8. 

The audio mixer 80 can also receive PCM audio input. PCM 
audio, also referred to as wave audio, is a most common form of 
audio used in CD-ROM applications. A PCM or "WAV" file is 
transferred digitally to the CS4232 audio mixer and processor 80 
under control by the CPU 42 which in turn decodes it through a 
16-bit DAC to an analog output before mixing it with the other 
possible audio sources for output to the system audio amplifier 
134. 

The system's sound generator 86, which is contained in the 
MIDI audio subsystem discussed more extensively below, produces 
an analog audio output routed to the CS4232 audio mixer and 
processor 80. 

The microphone 84 can be connected to the system for 
recording, karaoke and other musical voice input applications. 
The microphone input is preferably routed through a pre-amp 13 0 
to match levels with the other audio sources. This analog audio 
input connects to the CS42 32 audio mixer and processor 80. The 



system should preferably permit recording of microphone input as 
well as pass-through to the audio mixer and processor 80- These 
mixer and processor features are preferably accoxamodated by the 
CS4232 mixer and processor 80, Recording can be accomplished 
through a 16-bit ADC with PCM sampling at 44.1, 22.05, and 11.025 
kHz. This encoding can be performed in the CS4232 mixer and 
processor 80. 

As an additional source of music audio information, the 
audio output of the modem 70 can- be played through the player ^s 
audio system so the user can hear a dial tone or busy signal when 
dialing up an on-line service, for example. -The modem audio 
output can optionally be routed through a pre-mixer 142 for 
mixing with alarm and bell tones from the CS4 045 controller 48, 
50, which can in turn be made audible by the audio mixer and 
processor 80. 

Bus Subsystem 

The local CPU bus 108 links the main processor CPU 42, 
graphics controller 52, main memory 46, the CS4041 controller 66, 
and the timers, clocks and DMA/Internet Controller 48, 50, and 
the co-processor bus 150 (see FIG. 5) with a preferred 50 MB/sec 
throughput. An Industry Standard Architecture (ISA) Bus 144 has 
a preferred 8 MB/sec throughput and is used to link the 37C666 
I/O controller 78, the audio mixer and processor 80 and the modem 
70. The bridge between the buses 108, 144 are preferably buffers 
146. The preferred buffers 146 are bi-directional, 8-bit, 
74LS245 type, with tri-state capabilities. The buffers 146 also 



preferably provide extra drive current to support the loading of 
the data and address buses by the peripherals. 

MIDI Audio Subsystem 
Referring to FIG. 5, the MIDI audio subsystem 38 generally 
5 provides the components for sound generation; the interface with 
the musical keyboard 10 and associated optional MIDI controls, 
such as foot pedals 98 and optional pitch bend and modulation 
controls 100; and monitoring and assurance of system integrity 
through various diagnostic procedures. The MIDI audio subsystem 
10 38 has a bus 150 with a preferred throughput of 50 MB/sec and is 
linked to the music information processing subsystem 104 through 
O a translator/bus master 162 that couples to the local CPU bus 108 
fU (FIG. 3) in the music information processor subsystem 104. 

The MIDI co-processor 94 provides central management of the 
15p MIDI audio subsystem 38. The MIDI co-processor 94 is preferably 
Q a high performance, integrated processor containing a 32 bit CPU 
f^l with a 32 bit bus, 1 KB of 4-way instruction cache, 4 KB of 
static RAM, two 32 bit high-speed DMA controllers, two serial 
ports with baud rate generation, a bus watchdog, a periodic 
20 interrupt/ system timer, an interrupt controller, and dual 8 bit 
parallel ports. Certain speed-sensitive, low latency CPU 
operations including keyboard scanning and interrupt handlers can 
be executed by code running in the instruction cache. The MIDI 
co-processor CPU 94 preferably has a built-in serial port 164 
25 which can be used in manufacturing quality assurance testing to 
verify the workings of the entire assembly. 
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The main function of the MIDI co-processor 94 is to off-load 
the main CPU 42 (FIG. 3) during time critical functions such as 
keyboard/ foot pedal scanning, audio processor sound generation 
control, MIDI data stream processing, and analog to digital 
conversion control for volume, balance, pitch bend, modulation, 
and keyboard after-touch. The MIDI co-processor 94 preferably 
has dedicated local DRAM and flash memory 166. The local flash 
memory 168 preferably stores diagnostic software, chip set 
initialization values, and a loader program for obtaining 
standard operating software from the hard drive which will run on 
MIDI co-processor local memory 166. 

Sound generation is preferably provided by a MIDI sound 
generator wave table synthesizer 86. The MIDI wave table 
synthesizer 86, controlled by the MIDI co-processor 94, converts 
MIDI commands into an analog audio output which is sent back to 
the audio mixer 80 for mixing with the other audio sources (see 
also FIG. 3) . 

The MIDI wave table synthesizer 86 preferably includes an 
EMU-8000 audio processor 151 from E-mu Systems. Preferably, 
digital samples of 128 different instrument sets are stored in a 
sample memory or voice RAM 152 of preferably 8MB, and up to 32 
MB. The samples are preferably loaded into the voice RAM 152 
from the hard drive 64 (FIG. 3) during system boot-up under 
routing control by the MIDI co-processor 94. 

A dual port memory controller 154 allows both the MIDI co- 
processor CPU 94 and the main processor 42 to directly access the 



voice RAM 152* The MIDI co-processor CPU 94 preferably has 
direct access to quickly load the sample data from the hard drive 
64. The audio processor 151 also preferably has direct access 
for playing samples with minimum load time delay. The dual port 
5 memory controller 154 is also configured to allow the audio 

processor 151 to access the memory 152 at 16 bits wide and to 
allow the MIDI co-processor 94 to access the memory 152 at 32 
bits wide. 

The wave table voice samples are read and processed by the 
10 audio processor 151 and digitally mixed into a digital audio 

output stream which is converted to a stereo* analog audio output 
by a digital-to-analog converter 156, such as a Crystal 
ni Semiconductor DAC. The signals may be cleaned up with filters 
11 158 and directed to the audio mixer 80 of the multimedia audio 
15D subsystem 34 (See also FIGS. 2 and 3) . The audio processor 151 
Q preferably has a dedicated 1 MB RAM memory 160 for storing 
fu instructions for further manipulation of the voice samples, such 

r — ^ 

as control of their envelope parameters. 

The MIDI music subsystem 38 also serves to scan and process 

20 input from the keyboard 10 and other MIDI devices. The musical 
keyboard 10 is preferably an 88 key, velocity sensitive keyboard 
by Fatar and is preferably accompanied by the foot pedal assembly 
98. The keyboard/foot pedal interface 161 is designed to scan 
the keyboard 10 at a very high rate for superior resolution when 

25 measuring key velocity. The keyboard interface 161 can view 

sixteen switches at one time. The foot pedals 98 are preferably 
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scanned with the same mechanism and at the same rate as the 
keyboard keys. 

The keyboard and foot pedal interface 161 can consist of 

* 

four 3 to 8 decoders and two 8 bit bus drivers. There are 88 
5 keys on the preferred keyboard 10, with two switches per key. 
Each of the preferably three pedals also have two switches, 
thereby totalling 182 switches to be scanned per pass. The MIDI 
co-processor 94 is arranged to view 15 key-switches per scan, 
thus 6 long-word (32 bit) reads are required for one complete 
10 scan. To simplify and reduce the overhead to the MIDI co- 

processor 94, each set of sixteen switches can be assigned a 
^f^^ unique word-sequential address. The MIDI co-processor 94 can 

support automatic bus sizing, so that if it is instructed to read 
O a long word (32 bits) and the device is one word wide (16 bits), 
1S3 the MIDI co-processor CPU 94 will automatically perform two 
O sequential word reads to obtain a total of 32 bits. 
m The scan electronics 96 preferably include a Motorola 68HC05 

J] Chip with an analog multiplexer that can also poll and process 
pitch bend and modulation controls 100 as well as an optional 
20 volume pedal 163 for adjusting to the overall volume of the 
output of the audio processor 86. 

The MIDI co-processor 94 is also preferably arranged to 
assist in start up procedures and to run diagnostic procedures on 
system hardware to ensure proper operation. On power-up, the 
25 MIDI co-processor 94 preferably performs the following duties; 

initializes the CS4041-4045 chip set 48, 50, 66 (FIG. 3); sets up 
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the graphics controller; down-loads the default sound wave table 
from the hard drive 64 to the dual port memory 154 using its 
built-in DMA controller; and sets up specific user default 
settings, such as volume, balance, and sound table. 

After the MIDI co-processor 94 is running its control 
program, it releases the main processor CPU 42, which then 
preferably boots an operating system off the hard drive 64, 

Remote diagnostics, including software updates and repairs, 
can be run from a central off-sight facility through the modem 70 
to aid in trouble-shooting. Because the diagnostics are stored 
in the MIDI co-processor local memory 170, these tests do not 
rely on X86 Windows 95 operating system functions. 

The MIDI co-processor 94 preferably has full access to all 
the same devices as the main processor CPU 42 and preferably has 
the capability to run full diagnostics on the entire system. 
Both the MIDI co-processor 94 and main processor CPU 42 have 
access to each other's memory; however, the MIDI co-processor 94 
preferably has priority over the main processor CPU 42. 
Preferably, the MIDI co-processor 94 and main processor CPU 42 
are also responsible for updates to each other's flash memory for 
bug fixes, improvements and enhancements. This is accomplished 
through communication through the translator bus master 162. 

The MIDI co-processor 94 communicates with the host main 
processor CPU 42 through two 8-bit wide mailboxes. When either 
the MIDI co-processor 94 or the main processor CPU 42 sets a bit 
within its mailbox to "1", an interrupt occurs and a specific 
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message is transmitted based upon which bit is set. Potential 
messages from the MIDI co-processor 94 to the main processor CPU 
42 can include error occurrences, input buffer status, variations 
to volume or balance, and the like. 

Main processor CPU 42 to MIDI co-processor 94 interrupts 
generally mean that there is data waiting to be processed. For 
example, if a bit set to "1" represents that a buffer in the main 
processor CPU 42 has data to be processed, the MIDI co-processor 
94 could use its high-speed DMA to move a pre-defined block of 
data to its own local address space for processing. Other bit 
sets could mandate other functions, such as retrieving the serial 
number from EEPROM 168. 

The MIDI co-processor's address space can be divided into 
multiple segments, including local DRAM, audio processor DRAM, 
mail box, Flash ROM, 64K PC/AT I/O space, 128 MB main processor 
CPU space, and 1 MB BIOS PC/AT space. Similarly, the main 
processor CPU space can also be divided into segments; the lower 
72 megabytes for standard PC/AT memory and BIOS space; the next 
56 megabytes divided between the mailbox and VGA controller; the 
following 128 megabytes address for MIDI co-processor private 
space; and the last 1 megabyte for system BIOS. 

Because there is a difference in bus architecture between 
the preferred Motorola MIDI co-processor 94 and Intel X86 
processor preferred for the main processor CPU 42, a conversion 
or translation must occur for proper communications between the 
processors. One of the main differences between processors is on 
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which byte lane the least significant byte is placed. Motorola 
places byte 0, address 0 on D31-D24; Intel places byte 0, address 
0 on D0-D7, Four 8 bit bi-directional transceivers connect as 
follows: 

Motorola Bytes Intel Bytes 

D31-D24 D0-D7 
D23-D16 Connected to D8-D15 

D15-D8 D16-D24 
D0-D7 D25-D31 

The X86 family uses separate byte enables for each byte 
addressed, while Motorola uses AO-Al, SIZO, SIZl to determine 
which byte is addressed using programmable logic. In order for 
communication to occur between processors, signals must be 
translated between byte enable and address/size. 

The two processors also follow a different protocol for 
interacting with memory and I/O devices. The Intel processor has 
separate instructions for accessing memory and I/O devices while 
the Motorola processor treats all memory and I/O devices in the 
same manner. To adjust for this, a separate 64K address space is 
preferably assigned in the MIDI co-processor address space to 
generate the appropriate I/O access timing and signals in the 
main processor address space so that when the MIDI co-processor 
accesses the address space that is reserved for main processor 
memory space, the appropriate memory access timing and signals 
are generated. When the main processor CPU 42 accesses its 
memory space reserved for accessing the MIDI co-processor 94, 
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only one set of timing and control signals needs to be generated. 
Should each CPU 42, 94 attempt to access the other's address 
space at the same time, priority is given to the MIDI co- 
processor 94 to avoid a deadlock condition. ^ When the main 

5 processor 42 attempts to access the MIDI co-processor address 
space, logic in the Field Programmable Gate Array (FPGA) 
generates a bus request to the MIDI co-processor 94 and waits for 
a bus grant before the access occurs. When the MIDI co-processor 
94 wants to access the main processor address space, logic in the 
10 FPGA generates a LREQO through the controller 66 (FIG. 3) and 
waits for a LGNTO to be returned from the controller 66 before 
completing the access. 

To ensure that the MIDI co-processor 94 has complete control 

^ over the main processor 42 for proper start-up and diagnostic 
15]^^ purposes, it is preferably arranged to directly control the main 

I processor reset line. 

iri The MIDI co-processor 94 also controls a write-secure EEPROM 

m 168 that contains information including the system serial number, 
date of manufacture, hardware model type and version, and user 
20 installation date for warranty purposes. Once it has been 

written to, only certain parts of the EEPROM 168 can be altered. 

Software Subsystem 
Referring to FIG. 6, the components of the musical 
instrument system controlled by the graphical user interface are 
25 linked through multiple levels of software in a generally known 
manner. The musical instrument system utilizes micro-controller 
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software 170 in the MIDI co-processor 94 and the touch screen 
controller 126 to manage the interaction with the co-processor 
hardware 171, including the musical keyboard 10, the keyboard 
pedals 98 and the MIDI sound generator 86, , The touch screen 
controller 126 manages the input of signals from the touch screen 
overlay 22, constituting micro-controller hardware 173. 

On a less dedicated level, the information sources of the 
CD-Rom drive 16, floppy disk drive 14 and -the hard drive 64 are 
controlled through storage device drivers 172 ♦ The graphics 
controller 52 (FIG. 3) and the flat panel display 20 are 
controlled by graphics device drivers 174. Input device drivers 
176 are used to communicate with the touch screen controller 126 
while communications device drivers 178 are used to interface 
with the parallel and serial ports 72, 128, modem 70 and MIDI 
ports 76. An audio device driver 180 establishes control of the 
multimedia audio processor 80. 

The various drivers are further managed by an operating 
systems layer 182, which is preferably the Windows '95 platform 
by Microsoft Corporation. Applications used with the musical 
information system are preferably linked to the operating system 
through a Win32 API 184 and a Virtual Synthesizer API 186 within 
an API layer 187* 

The applications layer 188 can include a player shell 
application 189, a loader application 190 and can further include 
applications 192, 194, presented such as through CD-Rom, for 
execution with the musical instrument system. 



The components of the base operating system layer 182, the 
device driver layer 177, the system hardware layer 175, the 
micro-controller hardware 173 and the co-processor hardware 171 
can be developed and operated according to conventional personal 
5 computer and Windows '95 technologies parameters and guidelines. 

The loader application 190 is preferably developed to manage 
loading and starting of the application programs after initial 
start-up procedures. Preferably, the initial start-up procedures 
include start-up of the main components to the virtual machine 
10 manager and file system manager 182; loading and initialization 
of all required device drivers 172, 174, 176v 178 and 180; and 
J initialization of the hardware devices in the system hardware 
U layer 175. The loader application 190 is preferably the default 
\: application on system start up. After initial start-up, the 
15^ loader application 190 preferably launches the player shell 
^ application 189 to establish the graphical user interface 
ill according to the invention. 

Oj The loader application 190 is also preferably developed to 

,g launch other applications 192 and 194 when started from the 
2 0 graphical user interface of the system as discussed more fully 
below with reference to FIG. 25. 

The virtual Synthesizer API 186 preferably contains program 
functions for receiving/sending individual MIDI data packets from 
the MIDI hardware; recording/playing timed streams of MIDI data 
25 packets from/ to the MIDI hardware; and handling all layering of 
musical instrument sounds. 
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The touch screen controller software 126 polls the touch 
screen overlay 22 at regular intervals and communicates touches 
on the touch screen to the operating system drivers in layer 177. 

A MIDI co-processor control program 191 preferably handles 
scanning and translating user input on the musical keyboard 10 
and pedals 98 into MIDI input messages, which are sent to the 
operating system drivers in layer 177, as well as translating 
MIDI output messages into commands to the audio processor 86, The 
MIDI co-processor control program 191 is also preferably 
responsible for boot-up tests and diagnostic routines which test 
the co-processor hardware and generate error* notifications, if 
necessary, and for downloading sample sets into the voice RAM 152 
(FIG. 5) from the main memory 4 6 (FIG. 3) via DMA operations 
through the mailbox hardware 162 (FIG 5) • 

The player shell application 189 provides the main graphical 
user interface for' operating the musical instrument system as set 
forth below with reference to FIGS. 7-27. The player shell 
application 189 is preferably written to load graphics from the 
hard drive through Win32 API functions which communicate through 
the base operating system's file system manager 182 to the device 
driver 172 in charge of the hard drive 64, and then presents them 
to the display 2 0 through Win32 API functions which communicate 
through the base operating system's Virtual Machine Manager 182 
to the device driver 174 in charge of the display 20. After a 
graphical control screen is displayed, the player shell 
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application 189 waits for user input from the musical keyboard 10 
or the touch screen 22, 

On input from the touch screen 22, the system reacts as 
follows: the touch screen scanner software 12 6 registers a 
5 contact with the touch screen 22 and a hardware interrupt is sent 
to the main processing unit 42 (FIG- 2) ; the input device driver 
176 responsible for the touch screen 22 retrieves the X,Y 
position of the user contact from the touch screen scanner 126; 
and an input message is formulated and passed through the 
10 operating system to the player shell application 189 through the 
standard input notification functions of the* Win32 API 184, and 
the player shell application 189 executes the system function 
corresponding to the user input, 
^ On input from the musical keyboard 10, the system reacts as 

IS^ follows: A NOTE ON or NOTE OFF is registered by the MIDI Co- 

processor Control Program 191, and the co-processor CPU 94 (FIG. 
Ill 2) formulates the note message into a MIDI data packet. A 
m hardware interrupt is sent to the main processing unit 42 (FIG. 
,n 2) , The communications device driver 178 responsible for the co- 
20 processor 94 reads the MIDI data packet from the co-processor 94. 
A MIDI input message is formulated and passed through the 
operating system to the player shell application 189 through the 
MIDI functions of the virtual synthesizer API 186, and the player 
shell application 189 executes the system function corresponding 
2 5 to the user input. 
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If an input from the musical keyboard 10 or the touch screen 
22 instructs the player shell application 189 to play music 
through the MIDI synthesizer 86 (FIG. 2), the player shell 
application 189 formulates a MIDI output message to be sent to 
the synthesizer 86. The player shell application 189 calls a 
function from the Virtual Synthesizer API 186 to pass the message 
on, which passes the MIDI data through to the MIDI router 
hardware via an operating system call to the correct 
communications device driver 178. 

The device driver 178 causes a hardware interrupt via the 
mailbox mechanism 162 to the MIDI co-processor CPU 94. In 
response, the MIDI co-processor CPU 94 reads the MIDI output 
message from the device driver 178 through functions in the MIDI 
co-processor control program 191. The co-processor control 
program 191 plays the MIDI data by interacting with the audio 
processor 151 to load and play the correct sample of the correct 
instrument sample set. 

Consolida ted Control Through Graphical User Interface 
According to an aspect of the invention, music information 
access and processing, as well as sound parameter control, are 
performed solely through a visual input means such as a graphical 
user interface with touch control. Preferably, a plurality of 
interactive screens presented by the graphical user interface 
facilitate control of the musical instrument system including 
instrument selection and parameter control and single screen 



ensemble control as well as the management of music information 
from the various available sources. 

The graphical user interface of the musical instrument 
system is preferably controlled' through a touch screen system to 
facilitate rapid and convenient adjustment of system parameters. 
This arrangement facilitates understanding by the player and 
enables "real time" adjustment by professionals, particularly in 
the midst of a performance. 

The graphical user interface can alternatively be controlled 
and manipulated by a mouse, joystick or other signal input 
system. While the preferred mode of actuation is a touch screen, 
it is contemplated that the graphic controls of the invention may 
have application in environments in, which a mouse or other user 
interface is involved. Thus, terms, such as touch, contact, 
press, click and manipulate are used interchangeably herein to 
refer to the actuation of the function represented by a graphic 
image by user action. 

As shown in FIG. 7, a control screen 200, which can serve as 
a control graphic, presented by the graphical user interface of 
the invention can include a standard set of "buttons" that are 
available on every screen throughout operation of the musical 
instrument system. These buttons can include an access bar 202 
and provide a plurality of submenu functions. The standard 
buttons can also include graphic images for metronome control 204 
and help 206. The images presented by the graphical user 
interface can be varied as to artistic screen design, colors and 



arrangement within the scope of the invention. To facilitate an 
understanding of the images presented by the graphical user 
interface and their operation, a set of generic sub-menus for 
commonly used functions, such as saving and loading will be set 
5 forth. 

As shown in FIG. 7A, the metronome control icon 204 can 
activate a subscreen 205 displaying control arrows 207 for 
controlling the beats per minute, control arrows 209 for 
controlling the beats per measure, and control arrows 211 for the 
10 relative volume of the metronome. 

As shown in FIG. 7, the solo instrument' selection screen 200 
D allows selection from, for example, one hundred, twenty-eight 
W MIDI instruments on a single screen by way of an instrument 
U selector area 208. Category icons 210 can run across the top of 
1S\J the instrument selector area 2 08. The selection of a category by 
contact with one of the category icons 210 can cause a display of 
individual instruments unique to each category on a plurality. 
If. such as eight, buttons 212 preferably positioned under the 

category icons 210. The individual instrument buttons 212 are 
20 preferably "radio buttons" in which turning one on turns another 
off, with no more than one button selected at a time. The system 
is preferably configured so that the selected instrument sound is 
immediately effected upon its selection. 

The solo instrument control screen 200 preferably includes a 
25 plurality of effects buttons, for example, for chorus 214, reverb 
216 and equalization 218. The buttons 214, 216, 218 can be 
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activated to present pull-down or pop-up screens to provide 
alternatives for controlling the effects. The selected mode of 
each effect is preferably available in the sound output of the 
system upon selection • 
5 The solo instrument control screen 200 preferably includes a 

button 220 for activating a demonstration of the sound of the 
selected instrument and effects. The demonstration button 220 
can activate the system to play a short example of a song using 
the currently selected instrument. The system is preferably 
10 configured so that pressing any button on the screen 200 will 
interrupt or stop the demonstration mode. 
O The solo instriiment control screen 200 preferably includes a 

IaI button 222 for activating a key change, with directional arrows 
Ml, 224 and a numeric indication 226 of the number of semitones 
15m transposed. 

r Referring to FIG. 8, along the access bar 202, a second 

- ^ 

JIJ control screen directed to controlling an ensemble of mixed 
]^ selected instruments is available by selecting a button 232 
^^j labeled, for example, MIXED. Upon selection, the screen image 
20 234 can present a plurality of boxes 23 6 or other graphic images 
to represent musical instrument layers in the ensemble. The 
plurality of layers, for example, six, can each correspond to a 
selected instrument and effect arrangement. The six possible 
instruments comprising the mix can be indicated by the boxes 23 6. 
25 The mixed instruments control screen preferably presents an 
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access bar 238 including options for sound, range, volume, key, 
pan and drums control. 

Under the sound selection mode, the selection of a layer by 
activating one of the layer boxes 23 6 presents an instrument 
selection screen 240, which is preferably similar to the solo 
instrument screen 208 (FIG. 7) discussed above. The graphical 
user interface is preferably arranged so that the instrument 
group icon 242 selected within the instrument selector screen 240 
is placed in the layer box 23 6. The particular instrument within 
the instrument category and its associated effects can be 
selected as discussed above in reference to FIG. 7. 

The user can change an instrument in the mix by touching a 
layer box 23 6. The instrument selection screen 240 can visually 
link to that layer (such as Alto Sax) preferably by highlights. 
The user then uses the instrument selector 240 in the same manner 
as with solo instrument but can preferably also select a NONE 
button, which simply clears that layer and inserts no instrument 
(see lower box 236) . The instrument selector screen 240 remains 
on the screen until a new layer is selected. When a non-selected 
layer with an assigned instrument is selected, the instrument 
selector 240 will display the instrument group and instrument 
currently assigned to that layer. The sound ensemble selected 
can be saved, such as to the hard drive, and a previously stored 
ensemble can be retrieved from an information source using the 
save and load buttons 228, 230. Further details are set forth 
below in reference to FIGS. 2 6 and 27. 



Referring to FIG* 9, the mixed instruments access bar 238 
preferably includes a button 244 to activate a key range 
selection screen for establishing the keyboard range for each 
layer on a single screen. The button 244 can be labeled RANGE • 
The key range selection screen 246 includes an adjustment 
graphic, such as a keyboard image 248, for selecting the range of 
keys on the musical keyboard that will produce an instrument and 
effect designated for a particular layer 250, as selected in a 
prior mixed instrument sound selection screen (FIG. 7) . 
Preferably, the key range selection screen 24 6 presents a 
keyboard graphic 248 corresponding to the si-ze of the musical 
keyboard used with the system, for example, an 88-key register as 
shown. A separate keyboard control graphic 248 is provided for 
each of the layers of the mixed instrument ensemble. The layer 
box 23 6 preferably displays the icon of the musical instrument 
group previously selected for the layer 250 and the keyboard 
selection screen 246 preferably presents the name of the 
particular instrument selected from within the musical instrument 
group adjacent the associated layer box 236 and keyboard 
adjustment graphic 248 so that the user can quickly identify the 
sound to be attributed to the selected keyboard range. 

The system can be configured to activate the keyboard range 
selection in a variety of ways. Preferably, the keyboard 
adjustment graphic 2 48 can be clicked or touched at the beginning 
of the desired range and dragged to the end of the desired range. 



The keyboard graphic 248 preferably highlights or provides some 
other instant visual indication of the range selected. 

The layer 250 to be designated can be identified to the 
system by contacting the layer box 236 or by contacting the 
corresponding keyboard adjustment graphic 248, The layer 250 
selected remains activated for adjustment until the layer box 236 
or adjustment graphic 248 for another layer is contacted. The 
system is preferably configured so that all layers are unmuted 
and the first layer 252 is set to the full range of the musical 
keyboard and is highlighted for keyboard range adjustment. 

The mixed instrument keyboard selection* screen 24 6 can also 
provide a control button 254 labeled for example SET VIA KEYBOARD 
for activating keyboard range selection through the musical 
keyboard itself (not shown) . The particular musical instrument 
layer is selected by contacting the layer box and the keyboard 
selection is entered by touching the control button 254. The 
control button 254 preferably highlights, signalling the user to 
strike two keys on the musical keyboard to set the upper and 
lower limits of the desired range. The control button 254 
lowlights, indicating the deactivation of the keyboard selection 
mode, and the keyboard adjustment graphic for the selected layer 
indicates the key range. The set via keyboard function is also 
preferably deactivated by contacting any other control button or 
image on the screen 246. 

The keyboard selection screen 246 preferably provides a 
mute control icon 256 for each layer. The icon 256 can be 



contacted to mute and restore the sound of the associated layer. 
Each mute control icon 2 56 preferably visually displays the sound 
status; the mute control icon 2 56 displays, for example, that 
layer 258 for harp has been muted, while the remaining mute 
control icons indicate that their respective layers are 
configured to produce sound. The mute control icons 256 are 
preferably configured so that any of them can be contacted 
without changing to the layer presently selected. 

When the adjustment graphic 248 is touched, it activates and 
highlights that layer, for example, layer 250. That layer 250 
remains active and highlighted until another* layer graphic is 
touched. The mute buttons 256 are an exception in that touching 
them does not activate the associated layer. 

The sound ensemble selected can be saved, such as to the 
hard drive (FIG. 2) , and a previously stored ensemble can be 
retrieved from an information source using the save and load 
buttons 228, 230 (see FIGS. 26 and 27). 

Referring to FIG. 10, the mixed instruments access bar 238 
preferably includes a button 258 to activate a volume adjustment 
screen or volume adjustment control graphic 2 60 for single screen 
adjustment of all layer volumes. The button 258 can be labeled 
VOLUME. The volume adjustment screen 260 includes an adjustment 
graphic, such as a volume slider 262, for adjusting the volume 
level for each musical instrument layer selected in a prior mixed 
instrument sound selection screen (FIG. 7) . Preferably, the 
volume slider 2 62 presents a movable marker 264 that appears at a 



touched location and preferably displays a level number. The 
volume level and the corresponding position of the marker 264 can 
also be controlled by contacting and holding the arrow icons 266 
at the ends of the volume slider 262, The volume slider 262 can 
also present a central marker 268 that can be contacted to reset 
the volume to a default level and position the marker 2 64 to a 
centered position. 

Each layer box 236 preferably displays the icon of the 
musical instrument group previously selected for the layer and 
the screen 260 preferably presents the name of the particular 
instrument selected from within the musical instrument group 
adjacent the associated layer box 23 6 and volume slider 2 62 so 
that the user can quickly identify the sound being adjusted for 
volume. 

A separate volume slider 262 is provided for each of the 
musical instrument layers of the mixed instrument ensemble. The 
layer to be designated can be identified to the system by 
contacting the layer box 23 6 or by contacting the corresponding 
volume slider 262. The layer selected remains activated for 
adjustment until the layer box 236 or adjustment graphic 262 for 
another layer is contacted. 

To adjust all levels, a user can contact an ADJUST ALL 
button 27 0. All instrument layers highlight, then the user may 
slide any one of the six markers 264, any of the up/down arrows 
266, or any one of the 6 reset markers 268, to simultaneously 
adjust all layers to the same volume level. The ADJUST ALL 
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button 270 preferably remains in effect and highlighted until 
pressed again. The volume control screen 2 60 preferably provides 
a mute control icon 256 for each layer that can be contacted to 
mute and restore the sound of the associated layer as discussed 
above (FIG. 8) . The mute setting selected in one subscreen for a 
layer remains in that state for other subscreens. 

The volume settings selected can be saved, such as to the 
hard drive 54 (FIG. 2), and a previously stored ensemble can be 
retrieved from an information source using the save and load 
buttons 228, 230. 

Referring to FIG. 11, the mixed instruments access bar 238 
preferably includes a button 272 to activate a key transposition 
screen 274 to adjust the relative position of all layers on a 
single screen. The button 272 can be labeled KEY SHIFT. The key 
transposition screen 274 enables the user to adjust the key of 
each musical instrument layer. The key transposition screen 274 
includes an adjustment graphic, such as a key slider 276, for 
transposing the key of the corresponding instrument layer 278. 
The key marker 280 can be contacted and moved and preferably 
displays a plus or minus number representing the number of 
semitones of transposition for each musical instrument layer 
selected in a prior mixed instrument sound selection screen 234 
(FIG. 8) . The transposition and the corresponding position of 
the marker 280 can also be controlled by contacting and holding 
the arrow icons 282 at the ends of the key transposition slider 
276. The key transposition slider 276 can also present a central 



indicator 284 that can be contacted to reset the key to a default 
value and position the marker 280 to a centered position* 

The layer box 236 preferably displays the icon of the 
musical instrument group previously selected for the layer 278 
and the key transposition screen 274 preferably presents the name 
of the particular instrument selected from within the musical 
instrument group adjacent the associated layer box 236 and key 
transposition slider 276 so that the user can quickly identify 
the instrument sound being transposed, 

A separate key transposition slider 276 is provided for each 
of the musical instrument layers of the mixed instrument 
ensemble. The layer 278 to be designated can be identified to 
the system by contacting the layer box 236 or by contacting the 
corresponding volume slider 276, The layer 278 selected remains 
activated for adjustment until the layer box or adjustment 
graphic for another layer is contacted. 

To adjust all levels, the user can click on a button 285 
labeled, for example ADJUST ALL, all instruments highlight, then 
the user may slide any one of the markers 280, any of the 12 
up/down arrows 282, or any one of the 6 reset markers 284 to 
simultaneously adjust all layers to the same key transposition* 
The ADJUST ALL button 285 remains in effect and highlighted until 
pressed again. 

The key transposition screen preferably provides a mute 
control icon 256 for each layer that can be contacted to mute and 



restore the instrument sound of the associated layer as discussed 
above (see FIG. 9) , 

The key adjustment selected can be saved, such as to the 
hard drive 64 (FIG, 2), and a previously stored ensemble can be 
retrieved from an information source using the save and load 
buttons 228, 230, 

Referring to FIG. 12, the mixed instrument access bar 238 
preferably includes a button 286, labeled, for example, PAN, for 
presenting a pan adjustment screen 288 for controlling the 
balance of instrument sound for each instrument layer between 
left and right loudspeakers of the system (not shown) . The pan 
adjustment screen 288 can provide control graphics similar to the 
keyboard range adjustment screen {FIG, 9), the volume adjustment 
screen (FIG. 10) and the key transposition screen (FIG, 11), 
including a plurality of instrument group layer boxes 23 6, 
preferably bearing instrument group icons, corresponding 
individual instrument labels and adjustment graphics, such as a 
pan slider 290, The pan slider 290 can include a moveable marker 
292, adjustment arrow icons 294 and a reset marker 296 that 
operate as discussed above with respect to the volume adjustment 
screen 260 (FIG, 10), The pan control screen can further include 
mute control icons 256 for muting and restoring individual 
instrument sounds and a button for simultaneously activating all 
layers for adjustment. 

The pan adjustment screen facilitates rapid adjustment of 
instrument group panning by presenting all the instrument groups 



and their pan controls on a single contact by first contacting an 
ADJUST ALL button 298. 

The pan adjustment selected can be saved, such as to the 
hard drive 64 (FIG. 2), and a previously stored ensemble can be 
retrieved from an information source using the save and load 
buttons 228, 230. 

Referring to FIG, 13, the mixed instruments access bar 238 
preferably further provides a button 300, marked DRUMS for 
- example, for presenting a drum sound selector screen 302. The 
drum selector screen 302 can present drum a selection area 304 
similar to the multiple musical instrument selector screen 
discussed above in reference to FIG. 8. The drum selector 
screen 302 preferably includes a plurality of drum group boxes 
3 06 for displaying drum group icons. When a drum group is 
selected, a subscreen 308 containing numerous drum voices within 
the drum group is preferably displayed. The drum voice is 
selected by contacting the desired drum voice button 309. 

The allocation of the selected drum voice to a particular 
key of the. musical keyboard can be accomplished in a number of 
ways. The drum selector screen 302 preferably provides a 
keyboard graphic 3 01 displaying the keys of the musical keyboard 
10 (FIG. 1) of the system. 

An "I" shaped key indicator 303 can be provided to allocate 
a drum voice to a particular key on the musical keyboard 10 (FIG. 
1) . The upper portion of the key indicator 303 can operate as a 
button to set the presently selected drum voice to the marked key 

48 



on the keyboard graphic 301, The position of the key indicator 
3 03 can be moved by contacting and dragging the key indicator at 
its base. 

The drum selection screen 302 is also preferably interfaced 
5 with the musical keyboard 10 (FIG. 1) so that the drum voice can 
be set to a particular key by striking the desired key after the 
drum voice has been identified on the drum selector screen 3 02. 

Once a key has been assigned a drum voice, the keyboard 
graphic 302 preferably displays an assigned key highlight 305 or 
10 other visual indication. 

The drum selector screen 302 can also provide a volume 
Q control 310 for adjusting the level of the drum sounds and a 
y demonstration button 312 for playing a sample of the selected 
|,^, drum voice. All allocated drum voices can be cleared by a CLEAR 
■^^m t)^tton 307. The drum voice can be assigned to keys established 
J"' by the general MIDI standard by pressing GMSET button 313. 
"rz The drum voice arrangement selected can be saved, such as to 

ly, hard drive 64 (FIG. 2), and a previously stored ensemble can 

a be retrieved from an information source using the save and load 
20 buttons 228, 230. 

Referring to FIG. 14, a STYLES button 314 presents a styles 
selection screen 316 for selecting and loading a style routine 
from a source selected from one of the information sources 
indicated by the source selection graphic 318. The styles 
25 selection screen 316 preferably includes graphic buttons to stop 
320, play 322, and start synchronized play 324 of the style 
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rhythm upon depression of a first musical keyboard key (not 
shown) , 

To load a style, the user can select a style 32 6 from the 
selected source. To listen to that style 32 6, the user can 
contact the play button 322 • 

The style selection screen 316 can further include 
transposition 328 and volume 33 0 sliders • Also, trigger buttons 
to invoke INTRO 332, FILL 334 and END VARIATIONS 336 in the style 
rhythm can be provided ♦ 

The source selection graphic 318 preferably includes buttons 
for selecting the source of style files. These sources can 
include file in system memory 3 38/ CD-ROM 34 0 or floppy disk 342 
and can be expanded to include other sources. 

As can be appreciated from the previous description of 
control screens, the graphical user interface of the invention is 
designed to provide consolidated control of the sounds produced 
through the musical keyboard-activated sound generator and 
provides a unified control for accessing and utilizing music 
related digital information from a variety of sources. Thus, the 
access bar 202 for the system preferably includes buttons for 
further categories, such as player, recording, settings, storage 
and applications for managing musical information from the 
various information sources provided by the system (See FIG. 2) . 

Referring to FIG. 15, the system access bar 202 further 
includes a button 344, marked for example PLAYER, for accessing a 
screen or control graphic 346 for selecting and playing digital 



information from a variety of sources. The player screen 346 
allows the user to control play of standard MIDI files from the 
hard drive, CD drive or floppy disk drive; CD audio tracks; 
recordings made on the piano and the like. . 
5 The source is determined by selectors in the source graphic 

348. The PIANO selection 350 can be used to play music 
information from the hard drive 64 and preferably defaults to a 
MIDI songs directory, but can access other directories. The 
source graphic 348 can also present options for compact disc 352 

10 and floppy disk 354, The screen 346 provides a control bar 355 
with operation buttons including stop 356, play/pause 358, 
5! continuous single play toggle 360, skip back to start of song 
362, skip forward to start of next song 364. The operation 
buttons can further include a track slider 3 66 which can be 

ISlI manipulated by moving the marker 3 68 or the direction arrow icons 
3 70. Rewind and fast forward can be accomplished through 
in manipulating the slider 3 68 reverse and forward. 

gt A file/ song can be selected by contact with a labeled button 

372 within a directory frame 374, which preferably highlights the 

20 track and loads it for playback. If there are more songs than 
slots in the directory frame 374, scroller bars 376 preferably 
appear to permit scrolling through the list. The directory frame 
3 74 also includes a directory title box 378 with a pop-down 
function to list other directories options. 

25 The player screen 34 6 can also permit transfer to a detail 

screen 380. The detail screen 380 can present the individual 
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tracks 3 82 within a MIDI song and the instrument voices occupying 
those tracks. The title of the song is preferably shown in the 
box above the tracks 382. Each track 382 preferably provides 
mute control icons 3 84 for selectively muting individual track 
instrument sounds • 

The player screen 346 also preferably provides a key change 
slider control 386 for transposing the entire file. The tempo of 
the song may be change through the functions available under the 
universal metronome control screen as discussed above in 
reference to FIG. 7A. If the system program detects karaoke 
lyrics within a selected MIDI file, a LYRICS' tab 388 can be added 
to the player screen 34 6. 

Referring to FIG. 16, a selection of the LYRICS tab 388 
presents a lyrics screen 390 displaying lyrics 392 preferably in 
large type, and readable from a distance. The song title 
currently playing is preferably displayed. The lyrics screen 390 
preferably continues to display the control bar 3 55 and presents 
the key change control slider 386. This feature can be used to 
provide words for use during performance and enables mid- 
performance key changes by the user. 

Referring to FIG. 17, the access bar 202 of the system 
further preferably includes an activation button 394 for 
accessing a record screen 398 and can be labeled, for example 
RECORD, The sequencer provided by the record screen 398 is 
preferably easy to use, while still giving control over 



individual tracks. It is possible to record any instrument voice 
(mix or single) into a sequence. 

A time frame 400 shows the current position in the piece. 
The user can place a current position marker 404 in a time bar 
5 406 by clicking and dragging the marker 404 to the desired 

location. Additionally, arrow buttons 408 on either side of the 
time bar 406 can move the current position marker 404 by, for 
example, a 100th of a beat. The buttons on a control bar 410 
include a stop 412, play/pause 414, start of song 416, next song 
10 418, rewind 420, and fast forward 422. Additionally, there are 
two toggle buttons: a record toggle button 424 and a synchro- 
■^f start button 42 6. 

In operation, pressing play/pause 414 when the sequence is 
H stopped causes the sequence to begin playing/ recording (depending 
ISfiJ on the state of the record toggle button 424) at the current 

position 4 00 and the following buttons become disabled: start of 
III song 416, rewind 42 0, and fast forward 422. Pressing play/pause 
rn 414 when the sequence is playing, pauses the sequence. Pressing 
p stop 412 stops the sequence, restores the current position marker 
20 4 04 to its original position before play was started and all 

currently disabled buttons are enabled- Pressing rewind 420/fast 
forward 422 moves the position marker 404 backward/ forward at a 
relatively high speed until the beginning/end of the song is 
reached, or the button is released. Pressing the record button 
25 424 will toggle the play button 414 function to either play or 

record over the currently selected track. Pressing the synchro- 



53 



start button 426 will toggle synchro-start to either activated or 
not. If synchro-start 42 6 is activated, pressing any key or 
pedal is equivalent to pressing the play/pause button; the note 
played is audible and will be recorded if the record button 424 
is on. 

There can be a plurality, such as four, single instrument 
tracks 428. When a single instrument track 428 is marked for 
recording, the current single instrument sound selected on the 
single instrument selection screen 200 (FIG. 7) is heard and is 
recorded onto the track. Each track 428 preferably has the 
following attributes and controls (visual order is not required 
by this lists order) : track number, track name, volume, set-to- 
current • 

The track is simply numbered 1-4. When the user presses on, 
for example, track 430, that track is marked for recording. 
Recording can only happen on one track, so any other track marked 
for recording is cleared. There is preferably always at least 
one track marked for recording. 

The track name area 432 shows the current track name, which 
can be changed by pressing on the name area 432. This preferably 
brings up an on-screen alpha-numeric keyboard (not shown) for 
entering the name. 

A relative volume toggle 4 34 applies an overall volume 
adjustment for the track 428. A mute button 436 is a toggle that 
either mutes or sounds the track for playback or record. A muted 
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track preferably cannot be recorded upon. A clear track button 
438 clears the associated track. 

A SET TO CURRENT button 440 changes all instruments on the 
track to the current single instrument. When recording, the user 
hears all tracks unless they have been muted. 

The screen 398 preferably provides a mixed instrument track 
442. The controls for the mix instrument track 442 are 
preferably identical to the single instrument track 428 in 
function, with the exception that the associated SET TO CURRENT 
instrument button 440 will change all instruments on the track to 
the current mixed instrument ensemble established in the 
previously discussed mixed instrument screen 234 (FIG, 8) . 

The screen 398 also provides a styles track 444. The 
controls for the styles track 444 are preferably the same as the 
single and mixed tracks 428, 442, except that the associated SET 
TO CURRENT instrument button 440 will change all styles on the 
track to the current style as established and discussed above 
with reference to FIG. 14. Activation of the styles track 444, 
preferably displays a style control graphic 446. The styles 
control graphic 446 includes an INTRO button 448. When this 
button is pressed, and the style is not currently playing, the 
intro for the current style will play starting at the next beat 
followed by the normal play for the style. An END button 450 is 
also provided. When the style is playing, the end for the 
current style will play. When a FILL button 452 is pressed, the 
fill for the current style will play starting at the next beat. 



The control graphic can also provide mute controls 453 for 
selectively muting the voices in the styles track 444. 

Three other general controls are preferably available on the 
record screen 398, A CLEAR ALL button 454 clears all tracks in 
the sequence. The recorded piece selected can be saved, and a 
previously stored recording can be retrieved from an information 
source using the save and load buttons 228, 230. 

Referring to FIG. 18, the main access bar 202 for the system 
can include a SETTINGS button 456 for activating a settings 
screen 458 having a submenu bar 460. The settings screen 458 can 
include a frame 462 accessed through a GENERAL button 464 for 
adjusting date and time information in the system. To change, 
user touches either window for time 466 or for date 468, and 
touches arrows up/down keys 470 to move the selected number up or 
down . 

The settings screen 458 can also present a key touch 
adjustment frame 472. A user may touch and slide indicator 474 
and/or use HARDER or SOFTER buttons 476 to move indicator 474. 

Referring to FIG. 19, the submenu bar 4 60 can have PEDALS 
button 478 for accessing a pedals control screen 480 to assign to 
each pedals a function of, for example, soft, sostenuto or 
sustain from a choice of functions on pop-up screens (not shown) 
for each control area 481. A RESET button 482 can be provided to 
restore the pedals to factory default settings. 

Referring to FIG. 20, the submenu bar 460 can have an INPUT 
button 484 for presenting an input settings screen 486 to 



identify MIDI input devices or accessories attached to the 
system, such as a volume pedal (not shown) . A pop-up list of 
input devices can be provided by a control area 487 for each 
input port. A reset button 488 'will restore the inputs to 
5 default settings. 

Referring to FIG. 21, the submenu bar 4 60 can include a 
printer settings activation button 490 for presenting a printer 
settings screen 492. Available printers are preferably displayed 
in a selection list 494. Scroll bars can also be provided. A 
10 print button 496 can produce a printed test sheet. An ADD button 
498 can bring up a further selection interface with Add, Done, 
and Cancel buttons (not shown) . 

Referring to FIG. 22, the submenu bar 460 can have a screen 
saver settings activation button 500. The associated control 
ISflJ screen 502 allows the user to control screen savers. The START 
AFTER button 504 defines the period of inactivity before the 
III screen saver is activated. STOP AFTER button 506 controls the 
^ji method by which the interface is restored once the screen saver 
:% is activated. The ANY ACTION setting of button 506 will 
2 0 interrupt the screen saver when any input is sensed. ADD button 
508 functions as the add function in the printer selection screen 
492 as discussed above with reference to FIG. 21. A START button 
510 can begin a demonstration of the screen saver selected. 

Referring to FIG. 23, the access bar 2 02 preferably includes 
2 5 a button 512 such as STORAGE for accessing a storage control 
screen 514. The screen 514 can present a directory frame 516 
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that includes a plurality of source selection buttons for 
accessing the hard drive 518, compact disc 520 and floppy disks 
522. The directory frame 515 preferably includes a title block 
524 and a directory 526 of file'titles with, a scroller 528 for 
moving through the list. 

The control screen 514 preferably has a free space indicator 
53 0 that displays the free space on either the hard drive or 
floppy drive depending upon the source selected. 

To rename a file or directory, the user can select the 
source 518, 520 or 522 and select the file/directory 524 or 526 
in which the file/directory highlights. The- user then hits the 
RENAME button 532. The screen preferably includes an alpha- 
numeric keyboard graphic 534 for allowing user input of 
alphanumeric information in connection with renaming function. 
To delete a file or directory, the user can select the source 
518, 520 or 522, and select the file/directory 524 or 526 in 
which the file directory highlights. The DELETE button 536 is 
then selected. Selection of the FORMAT button 53 5 executes a 
format program on an inserted floppy disk. 

To copy a file or directory to another destination, the user 
selects the source 518, 520 or 522 directory, and selects the 
file/directory 524 or 526. The user then hits the COPY button 
538. Referring to FIG. 24, a second selection-type interface 540 
preferably pops up to the right of the first frame 516. The user 
can then select the location for the file/directory 544 or 546 to 
be copied to and hit the COPY button 548 or cancels the copy by 
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touching the CANCEL button 550. A similar screen can be provided 
for move function activated by the MOVE button 552 (FIG, 23)* 

Referring to FIG. 25, the system access bar 202 can provide 
an APPLICATIONS button 554 to present a control screen 555 for 
starting other applications 556 that may be used in the musical 
instrument system environment. The source of applications can be 
selected with a hard drive selection button 558, CD selection 
button 560 or a floppy disk selection button 562. 

To begin an application, the user can select one of the 
application icons 556, which preferably highlights, and click the 
START icon 564. The player shell application 189 (FIG. 6) is 
closed, and the selected application runs. Through this feature, 
the musical instrument system provides a music information 
environment in which newly developed applications can be readily 
loaded and can utilize an integrated musical keyboard and various 
music information sources through a readily accessible, 
consolidated graphical user interface. 

Referring to FIG. 26, the SAVE button 228 presented in the 
various control screens (see FIGS. 7-24) can present a further 
control screen 590 providing a source selection graphic 592 
including buttons for hard drive 594, compact disc 596 and floppy 
598, as well as a directory or folder frame 600 listing the 
current folder and a pull down control for additional directories 
available as storing locations. New directories can be added by 
touching an activation button 602 labeled, for example, NEW 
FOLDER. A list of files 601 in the current folder 600 can be 



displayed, and the current file can be highlighted. The current 
file can be a file previously loaded, changed and now presented 
for re-saving. In this circumstance, the system preferably 
defaults the current file name in the identification area 603 for 
saving. The identification area 603 can provide a RENAME 
control, which can activate a further control area having an 
alpha-numeric keyboard graphic 604 for entering a new name. The 
name of the file is preferably displayed in a frame 606 above the 
alpha-numeric keyboard graphic 604. The new name in frame 606 
can approved upon pressing the OK button 608 or can be canceled 
through a CANCEL button 610. Errors in the new name can also be 
corrected by a CLEAR button 607. 

Upon entry of a new name, the name is changed (not shown) in 
the identification area 603. The file can then be saved by 
actuation of a SAVE button 609 or the process can be terminated 
with a CANCEL button 611. 

For newly created recordings or ensemble settings or other 
information to be recorded for the first time, rather than 
modification of a retrieved file, the alphanumeric screen 604 
preferably appears first for entry of the new file name 606. The 
save graphic 590 and the alphanumeric area 604 can appear 
simultaneously or separately on the display. 

Referring to FIG. 27, previously stored settings or files 
can be retrieved through actuation of the LOAD button 230 present 
on the various control screens (see FIGS. 7-24). The LOAD button 
can activate a subscreen 612 displaying a source selection 
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graphic 614 including hard drive 616, compact disc 618 and floppy- 
disk drive 620 as well as a directory frame 622 listing the 
directory and a scrollable list of files. Once the desired file 
has been selected through contact and preferable highlight, the 
file can be loaded by touching the LOAD button 624 or the 
operation can be canceled through the CANCEL button 626. 

The foregoing description is directed to various embodiments 
and preferences for using the features of the invention. Various 
alternatives not specifically discussed but within the scope of 
the invention may now be apparent to one having ordinary skill in 
the art. Accordingly, the scope of the invention should not be 
limited by the above discussion, but rather should be determined 
by a reasonable construction of the following claims. 



We claim: 

1. An electronic music instrument system, comprising: 
an electronic music instrument, having selectable groups of 
reproducible sounds and individually selectable reproducible 
sounds within said groups of sounds; 
5 an audio signal generator for energizing an audio amplifier 

responsive to different ones of a plurality of digital audio 
sources, including said groups of sounds and said individual 
sounds of said electronic music instrument; 

a graphical user interface for displaying at least one 
10 control graphic representing controllable parameters of said 
audio signals generated by said generator; and, 
JJJ control means responsive to operation of said control 

graphic for adjusting said controllable parameters of said 
aj generator and for selectively coupling different ones of said 
15:. sources to said generator. 

ill 2, The system of claim 1, wherein said graphical user 

m interface comprises: 

■ f: a video display; and, 

a touch-responsive overlay. 

3. The system of claim 2, wherein one of said controllable 
parameters is voice selection for single instrument sound. 

4. The system of claim 3, wherein one of said controllable 
parameters is key transposition for said single instrument sound. 

5. The system of claim 4, wherein one of said controllable 
parameters is effects selection for said single instrument sound. 
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6. The system of claim 2, wherein: 

said controllable parameters include multiple instrument 
sound selection and sound layer assignment, controlled responsive 
to operation of said at least one control graphic; and, 
5 said at least one control graphic is displayed on a single 

screen of said graphical user interface. 

7. The system of claim 6, wherein: 

said musical instrument comprises a musical keyboard; and, 
one of said controllable parameters is allocation of each said 
sound layer to keys on said musical keyboard, said allocation 
5 also being controlled responsive to operation of said at least 
y one control graphic. 

8. The system of claim 6, wherein: 

H said musical instrument comprises a musical keyboard; and, 

fy operation of said at least one control graphic also enables key 

allocation responsive to said musical keyboard. 
HI 9. The system of claim 6, wherein: 

'ff, said graphical user interface generates a further control 

^ graphic; 

one of said controllable parameters is a volume control for 
5 each said sound layer, responsive to operation of said further 
control graphic; and, 

said further control graphic is displayed on a further 
single screen of said graphical user interface. 
10. The system of claim 6, wherein: 
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said graphical user interface generates a further control 
graphic; 

one of said controllable parameters is key transposition 
adjustment for each said sound layer, responsive to operation of 
5 said further control graphic; and, 

said further control graphic is displayed on a further 
single screen of said graphical user interface. 

11. The system of claim 6, wherein: 

said graphical user interface generates a further control 
graphic; 

one of said controllable parameters is pan adjustment for 
^ each said sound layer, responsive to operation of said further 
1^ control graphic; and, 

J said further control graphic is displayed on a further 

single screen of said graphical user interface. 

12. The system of claim 6, wherein: 

said musical instrument comprises a musical keyboard; and, 
S one of said controllable parameters includes allocation of 

.fl keys on said keyboard to different sounds, responsive to 
operation of said at least one control graphic. 

13. The system of claim 6, wherein: 

said musical instrximent comprises a musical keyboard; and, 
one of said controllable parameters includes assignment of 

said sound layers to ranges of said keys, responsive to operation 

of said at least one control graphic. 
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14. The system of claim 6, wherein said controllable 
parameters include drum syncopation, triggers, key change and 
volume, responsive to operation of said at least one control 
graphic. 

15. The system of claim 6, wherein: 

said graphical user interface displays a further control 
graphic; 

said controllable parameters include multiple channel 
recording and allocation of different ones of said sound layers 
to different ones of said multiple channels, responsive to 
operation of said further control graphic; and, 

said further control graphic is displayed on a further 
fy single screen of said graphical user interface. 
^ 16- The system of claim 15, wherein said controllable 

g parameters include allocation of drum syncopation to different 
^ ones of said multiple channels, responsive to operation of said 
further control graphic. 

17. The system of claim 16, wherein said controllable 

C parameters include same-channel recording and different-channel 
recording of said sound layers and said drum syncopation, 
responsive to operation of said further control graphic. 

18. The system of claim 2, wherein: 

said musical instrument comprises a musical keyboard; 

said controlled parameters include drum voice selection and 
allocation to a key on said musical keyboard, responsive to 
operation of said at least one control graphic; and, 
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said at least one control graphic is displayed on a single 
screen of said graphical user interface. 

19. The system of claim 2, wherein: 

said controllable parameters include drum syncopation, 
triggers, key change and volume, responsive to operation of said 
at least one control graphic; and, 
5 said at least one control graphic is displayed on a single 

screen of said graphical user interface. 

20. The system of claim 1, wherein: 

said musical instrument comprises a musical keyboard; and, 
^ said controllable parameters include audio characteristics 

of further sources, other than said sources originating in said 
5fi| musical keyboard of said musical instrximent. 

"^^^ system of claim 20, wherein said further sources 

comprise: 
^ a hard disk drive; 

%\ ' a floppy disk drive; and, 
5"^ a compact disc drive, 

22. The system of claim 2, wherein said plurality of 
sources further comprises a modem for communicating with an on- 
line source of digital information. 

23. The system of claim 2, further comprising a piano 
housing, said system being mounted within said piano housing. 

24. The system of claim 2, further comprising a piano 
housing, said video display being mounted in said piano housing. 
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i 

25. The system of claim 2, further comprising a piano 
housing having a music stand formed integrally therewith, said 
video display being mounted in said music stand, 

26. The system of claim 2, further comprising a piano 
housing having a music stand formed integrally therewith, said 
video display forming an integral part of said music stand. 

27. The system of claim 1, wherein said control means is 
exclusively responsive to said graphical user interface. 

28. The system of claim 2, wherein said electronic musical 
instrument comprises a musical keyboard. 

29. The system of claim 28, wherein said control means is 
^ exclusively responsive to said graphical user interface and to 
ry said musical keyboard. 

30. The system of claim 1, wherein said sources include at 
li least one prerecorded source. 
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Abstract 

An electronic music instrument system, comprising: an 
electronic music instrument, having selectable groups of 
reproducible sounds and individually selectable reproducible 
sounds; a signal generator for energizing an audio amplifier 
responsive to different digital audio sources, including the 
groups of sounds and the individual sounds a graphical user 
interface for displaying at least one control graphic 
representing controllable parameters of the audio signals 
generated by the generator; and, a controller responsive to 
operation of the control graphic for adjusting the controllable 
parameters of the generator and' for selectively coupling 
different ones of the sources to the generator. The graphical 
user interface can comprise: a video display; and, a touch- 
responsive overlay. The controllable parameters can include 
multiple instrument sound selection and sound layer assignment, 
controlled responsive to operation of the at least one control 
graphic. The at least one control graphic can be displayed on a 
single screen. 
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I hereby state that I have reviewed and understand the contents of the above 
identified specification^ including the claims, as amended by any amendment referred 
to above. 

I acknowledge the duty to disclose information which- is material to the 
examination of this application in accordance with Title 37, Code of Federal 
Regulations Section 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code 119 
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